Papers by Mitchell A Gordon
Data and Parameter Scaling Laws for Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that supervised neural machine translation models scale like a power law with the amount of training data and number of non-embedding parameters in the model. |
| Approach: | They show that cross-entropy loss of supervised neural machine translation models scales like a power law with the amount of training data and number of non-embedding parameters in the model. |
| Outcome: | The proposed model can predict BLEU and ROI of labeling data in low-resource language pairs. |